Field of the Invention
The present invention relates to program code development and more particularly to capturing performance data for newly developed or changed code.
Description of the Related Art
Software testing is an investigation conducted to determine the quality of a software product or service under test. Software testing can be performed by the programmer or team of programmers who had developed the software program code of the software product in the first instance, or for more organized efforts, by a group of dedicated software testers or quality assurance testers. The testers typically use different types of testing methods, including white box testing, black box testing, and gray box testing. White box testing refers to when a tester has access to the internal data structures and algorithms including the computer code that implements the structures and algorithms; black box testing treats the software as a “black box”—without any knowledge of internal implementation; and, gray box testing involves the tester having knowledge of internal data structures and algorithms for the purpose of designing test cases, but testing at the user, or black-box level.
During the development phase for a new release of an application under test, part of the testing of a new code base of the application under test involves capturing performance data, such as central processing unit (CPU) usage resulting from the application under test. Performance measurement usually involves running sets of well-defined test cases against the application under test and using system tooling to measure concomitant changes to the system performance resulting from the execution of the application under test when compared to earlier releases of the application under test. In addition, software monitoring products can generate additional data from the execution of the application under test, which can provide a coarse level of CPU-usage at the region level and at an individual transaction level.
But, a tester using software monitoring tools to test an application under test still must use a “post-processing tool” to analyze and breakdown performance data collected during testing. In addition, a tester still must run specific benchmark comparisons on different computing systems to compare and then contrast CPU costs for code paths used for specific workloads in testing an application under test. Oftentimes, this level of in-depth performance analysis and testing is not available to support and service software developers deploying changes to an application under test. Thus, software developers often must apply small alterations to existing programs and work to schedules that preclude complete performance reevaluations of an application under test after each of the changes have been applied to the application under test. With no analysis and testing tools available, performance metrics thus are only based upon the experience and best judgment of the software developer.